7 research outputs found
Distance-based Analysis of Machine Learning Prediction Reliability for Datasets in Materials Science and Other Fields
Despite successful use in a wide variety of disciplines for data analysis and
prediction, machine learning (ML) methods suffer from a lack of understanding
of the reliability of predictions due to the lack of transparency and black-box
nature of ML models. In materials science and other fields, typical ML model
results include a significant number of low-quality predictions. This problem
is known to be particularly acute for target systems which differ significantly
from the data used for ML model training. However, to date, a general method
for characterization of the difference between the predicted and training
system has not been available. Here, we show that a simple metric based on
Euclidean feature space distance and sampling density allows effective
separation of the accurately predicted data points from data points with poor
prediction accuracy. We show that the metric effectiveness is enhanced by the
decorrelation of the features using Gram-Schmidt orthogonalization. To
demonstrate the generality of the method, we apply it to support vector
regression models for various small data sets in materials science and other
fields. Our method is computationally simple, can be used with any ML learning
method and enables analysis of the sources of the ML prediction errors.
Therefore, it is suitable for use as a standard technique for the estimation of
ML prediction reliability for small data sets and as a tool for data set
design
Identification of high-reliability regions of machine learning predictions in materials science using transparent conducting oxides and perovskites as examples
Progress in the application of machine learning (ML) methods to materials
design is hindered by the lack of understanding of the reliability of ML
predictions, in particular for the application of ML to small data sets often
found in materials science. Using ML prediction for transparent conductor oxide
formation energy and band gap, dilute solute diffusion, and perovskite
formation energy, band gap and lattice parameter as examples, we demonstrate
that 1) analysis of ML results by construction of a convex hull in feature
space that encloses accurately predicted systems can be used to identify
regions in feature space for which ML predictions are highly reliable 2)
analysis of the systems enclosed by the convex hull can be used to extract
physical understanding and 3) materials that satisfy all well-known chemical
and physical principles that make a material physically reasonable are likely
to be similar and show strong relationships between the properties of interest
and the standard features used in ML. We also show that similar to the
composition-structure-property relationships, inclusion in the ML training data
set of materials from classes with different chemical properties will not be
beneficial and will slightly decrease the accuracy of ML prediction and that
reliable results likely will be obtained by ML model for narrow classes of
similar materials even in the case where the ML model will show large errors on
the dataset consisting of several classes of materials. Our work suggests that
analysis of the error distributions of ML predictions will be beneficial for
the further development of the application of ML methods in material science
Self-Organizing Maps Algorithm for Parton Distribution Functions Extraction
We describe a new method to extract parton distribution functions from hard
scattering processes based on Self-Organizing Maps. The extension to a larger,
and more complex class of soft matrix elements, including generalized parton
distributions is also discussed.Comment: 6 pages, 3 figures, to be published in the proceedings of ACAT 2011,
14th International Workshop on Advanced Computing and Analysis Techniques in
Physics Researc